Ensembles of Nearest Neighbours for Cancer Classification Using Gene Expression Data

It is known that an ensemble of classifiers can outperform a single best classifier if classifiers in the ensemble are sufficiently diverse (i.e., their errors are as much uncorrelated as possible) and accurate. We study ensembles of nearest neighbours for cancer classification based on gene expression data. Such ensembles have been rarely used, because the traditional ensemble methods such as bagging and boosting are unable to inject diversity into nearest neighbour classifiers. To alleviate this problem, feature selection prior to classification is done. After that, diversity-based ensemble pruning before classifier combination can lead to an ensemble outperforming a single best nearest neighbour. However, such a result cannot always hold as demonstrated by experiments with two gene expression datasets. The reason seems to lie in dataset complexity. We show that if a dataset is easy for a single nearest neighbour to accurately classify, then an ensemble of nearest neighbours is hardly capable to get it better. In contrast, if a dataset is difficult according to certain complexity measures, then an ensemble of nearest neighbours is superior in performance to a single best nearest neighbour.